Monocular Fisheye Camera Depth Estimation Using Semi-supervised Sparse Velodyne Data

Near-field depth estimation around a self-driving car is an important function that can be achieved by four wide-angle fisheye cameras having a field of view of over 180◦. CNN based depth estimation produce state of the art results, but progress is hindered because depth annotation cannot be obtained manually. Synthetic datasets are commonly used but they have limitations. For instance, they do not capture the extensive variability in the appearance of objects like vehicles present in real datasets. There is also a domain shift while performing inference on natural images illustrated by many attempts to handle the domain adaptation explicitly. In this work, we explore an alternate approach of training using sparse LIDAR data as ground truth for depth estimation for fisheye camera. We built our own dataset using our self-driving car setup which has a 64-beam Velodyne LIDAR and four wide angle fisheye cameras. LIDAR data projected onto the image plane is sparse and hence viewed as semi-supervision for dense depth estimation. To handle the difference in view-points of LIDAR and fisheye camera, an occlusion resolution mechanism was implemented. We started with Eigen’s multiscale convolutional network architecture [1] and improved by modifying activation function and optimizer. We obtained promising results on our dataset with RMSE errors better than the state-of-the-art results obtained on KITTI because of vast amounts of training data. Our model runs at 20 fps on an embedded platform Nvidia TX2.